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SOFT BIT COMPUTATION FOR A REDUCED STATE EQUALIZER 

Technical Field of the Invention 

This invention relates generally to digital communication, and more 
particularly to low complexity equalization. 

Background Art 

For Time Domain Multiple Access (TDMA) wireless communications 
systems, an important problem is to mitigate intersymbol interference (ISI) caused 
when data is transmitted on a dispersive channel with accompanying background 
noise. This problem is different depending upon what type of air interface is 
employed. In a second generation system such as the global system for mobile 
communications (GSM) it suffices to treat the ISI problem in ways that are not 
adequate for third generation air interfaces such as EDGE (enhanced data rates for 
GSM evolution). In GSM, optimum performance in terms of error probability is 
typically accomplished by maximum likelihood sequence estimation (MLSE), but 
MLSE detection cannot be used to perform optimal equalization in EDGE, due to 
changes in both the modulation and the channel coding. 

When discrete pulses or bursts of information are transmitted through a 
dispersive medium, they travel over multiple propagation paths to the receiver, and 
the information transmitted to the receiver will appear smeared. These multipath 
effects commonly result when signals are reflected to the receiver. The process 
implemented on the received signal to mitigate ISI is known as "equalization." This 
equalization process estimates the individual data symbols which were originally 
transmitted. 

Various approaches have been taken to the equalization problem; linear 
equalization and decision-feedback equalization (DFE) are two common equalization 
strategies. However, DFE has poor performance in weak channel coding schemes, 
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such as EDGE transmission modes MCS8-9. A somewhat better approach has been 
based on MLSE, using the well-known Viterbi algorithm, although this approach 
ahs drawbacks as explained below. See Shah (U.S. Patent No. 6,134,277); Cooper 
(U.S. Patent No. 5,502,735). 
5 Those skilled in the art will understand that a communication channel 

behaves in many respects like a filter, and therefore ISI itself may be modelled as a 
filter. The length of the filter is the extent of ISI, and this length may be referred to 
as the impulse response length L. Unlike DFE, MLSE detection typically uses 
i ^ trellis diagrams which show a progression of states with the passage of time, and the 

^| 10 number of states S is given by M to the power of L-l (i.e. S=M L " 1 ) where M is the 
flj number of symbol levels in the data alphabet used. Consequently, for large 

J response lengths and large alphabet sizes, the equalizer implementation becomes 

m 

Irz immensely complex. The complexity of the Viterbi Algorithm increases 

;L exponentially with channel length, and therefore implementation of the full Viterbi 

fg 15 Algorithm becomes unacceptably complex in EDGE. Thus, MLSE using the full 
y Viterbi algorithm is inadequate, and a sub-optimal equalizer capable of performance 

p comparable to that of MLSE is necessary, with acceptable low complexity. 

A well-known sub-optimal type of equalizer employs Reduced State 
Sequence Estimation (RSSE), wherein a reduced-state maximum likelihood (ML) 
20 trellis uses set partitioning and decision feedback techniques. See, for example, 

"Reduced-State Sequence Estimation With Set Partitioning and Decision Feedback" 
by M. Eyuboglu, and S. Qureshi, IEEE Trans. Comm., vol. 36, pp. 12-20, January 
1988. A particularly useful special case of RSSE is Delayed Decision Feedback 
Sequence Estimation (DFSE). A precondition for applying RSSE is that the input 
25 signal is of minimum phase (i.e. the impulse response power decays in time), and 
this condition can be satisfied by applying prefiltering to the received signal, for 
example using a DFE feedforward filter. 

Additional equalization reliability measures are helpful in order to improve 
channel decoding subsequent to equalization, and for this purpose it is desirable to 
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generate soft-output information. Soft bits contain important reliability information 
that greatly aids the decoder located within the receiver downstream from the 
equalizer. 

Soft-output information can be produced by an algorithm called symbol-by- 
5 symbol Maximum A Posteriori (MAP) decoding, which can calculate the probability 
of each symbol given the whole sequence that has been received. For example, see 
"Reduced State Soft-Output Trellis-Equalization Incorporating Soft Feedback," by 
Muller et al., Proc. IEEE GLOBECOM '96, November 1996. A max-log-MAP 
algorithm is familiar to persons skilled in the art as a simplified version of a MAP 

10 algorithm, and max-log-MAP equalizers operate in a logarithmic domain as 

compared to a probability domain. Another known method for producing soft- 
output information is using decision feedback partial sequence estimation (DFPSE). 
DFPSE equalizers are equalizers which employ a non-zero delay reduced state 
trellis calculation and which achieve reduction of implementation complexity by (1) 

15 employing decision feedback, (2) relying on improved accuracy of initial symbol 
detection which reduces the scope of searching, and (3) using ML or Max-log-MAP 
detection criteria. DFPSE is also known as block decision feedback equalization. 
See "Block Decision Feedback Equalization" by Williamson et al., IEEE Trans 
Comm., vol. 40, pp. 255-264, February 1992. As will be understood by those of 

20 ordinary skill in the art, non-zero delay refers to the those situations in which, after 
a signal is received, there must be a waiting period before soft bits are computed, 
and this may be due to the need to examine more than one received symbol. 

Although using a reduced number of states "S" (e.g. via RSSE or DFPSE) 
produces very good hard symbol decisions with extremely low complexity, 

25 problems arise in producing soft decisions (i.e. soft bits) such as log likelihood 

ratios or bit probabilities of transmitted bits. A familiar response to these problems 
is to use trellis truncation, but trellis truncation in and of itself does not allow 
calculation of MAP or MLSE soft decisions for all bits of a symbol. In other 
words, when the number of states in the equalizer is reduced to obtain an acceptably 
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low complexity, computing soft bits for all of the bits becomes problematic 
according to existing methods. 

The total number of states in a subset trellis is given by the product of the Jk 
and each Jk ranges from 1 to M. So, the minimum number of states for an equalizer 
5 would be 1 for J = { 1 , 1 , 1 . . . 1 } which actually corresponds to DFE. At the other 
extreme is full MAP or MLSE in which case J = {Ji, J2, J3.. Jl-i} and Jj = M and 
thus the number of states would be M L1 as discussed above. Recall that W L" is 
impulse response length, and also bear in mind that the Jk are subject to a constraint 
Ji > h > h > ... Jl-i. For an equalizer having J = {Ji, J2, J3...JK-1, 1, 1...1} where 
10 K< L and Jj < M the hard decision performance remains extremely good, but 
calculation of adequate soft decisions is difficult. Therefore, a need exists to 
increase reliable performance of low complexity equalizers, especially for high 
N- order modulations, by more fully and successfully producing soft bit information, 

p In general, EDGE systems involve capture of 2 or 3 taps in the MAP or 

r? 15 MLSE part of the equalizer, and this means 8 or 64 states respectively for existing 
yj| DFSE equalizers, or alternatively 4 or 16 states respectively for existing RSSE/RS- 

hk MAP equalizers with 4-way set partitioning. Digital signal processing (DSP) 

implementation of the simplest 4-state equalizer currently requires some 32 million 
instructions per second (MIPS) per time slot for EDGE at 8 PSK (phase shift 
20 keying). In order to provide higher data rate, it is desirable to have lower 

complexity algorithms than are currently available, or else DSP will be too costly. 
Although 2-way set partitioning with RSSE or RS-MAP is a candidate for achieving 
lower complexity, it entails problematic soft bit computation. Likewise, soft symbol 
equalization is a candidate for replacing soft bit equalization, but soft symbol 
25 equalization is approximately 1 .5 dB worse than soft bit equalization for 8PSK 
EDGE, and this situation gets even worse for higher order modulations using 
current technology. 
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Disclosure of the Invention 

The present invention provides a novel method of computing soft bits (which 
hereinafter refers to log likelihood ratios or bit probabilities) for any reduced state 
equalizer. This new method for making soft decisions uses low-complexity, while 
5 providing soft bit information for substantially all received bits. This invention 
allows an equalizer to be implemented with a minimum number of states, while 
retaining high performance. This reduction of implementation complexity is 
accomplished by methods which, for example, combine MLSE or MAP soft 

Q decisions with either decision-feedback equalization (DFE) decision-making or with 

ill 10 other zero-delay soft decision-making. 

m The method involves computing a first set of soft bits for transmitted bits in 

a received signal, using a reduced-state trellis with finite non-zero delay, calculating 
H hard decisions in response to the received signal, and also ensuring that substantially 

p all soft bits are computed by employing zero-delay soft decision-making or decision- 

2 15 feedback equalization to compute a second set of soft bits. Furthermore, the hard 

decisions are used to compute the second set. 
M This invention assures that the number of states in the equalizer is reduced to 

obtain an acceptable complexity, while also ensuring that soft bit computation is not 
omitted for a substantial number of bits. Therefore, this method increases the 
20 reliability of low complexity equalization, especially for high order modulations, by 
more fully and successfully producing soft bit information. 

According to the invention, soft bits can be computed from a received signal 
using a reduced-state trellis with MAP soft decisions, and hard decisions are also 
computed in response to the received signal. Zero-delay soft decision-making or 
25 decision-feedback equalization is also employed, using the hard decisions to 

compute the soft values, and this ensures that soft bits are computed for substantially 
all bits. Decision-feedback equalization, using these hard decisions to compute the 
soft values, is unconventional and provides a marked improvement in soft bit 
computation. 
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The present method is flexible in that it can be used in a number of different 
equalization environments. For example, the trellis calculation may produce 
maximum a posteriori (MAP) soft decisions, and/or may produce soft decisions with 
reduced state sequence estimation (RSSE), reduced state MAP, and/or reduced state 
5 MLSE. The trellis calculation may utilize maximum likelihood (ML) techniques, 
and may produce soft decisions with decision feedback partial sequence equalization 
(DFPSE). 

The present soft bit equalization invention not only offers a 1.5 dB gain (or 
better) as compared to soft symbol equalization, but also allows the number of states 

10 in the equalizer to be reduced to the minimum possible value. Note that different 
types of equalizers are not necessarily altogether different and distinct from each 
other, and there is often considerable commonality within subsets of equalizers. 
The present method is applicable not only to various types of equalizers, but also to 
arbitrary modulation sizes, and to any multiple access schemes such as 

15 TDM A/CDMA requiring a reduced state equalizer. 

Preferable implementation modes for the present invention include digital 
signal processor (DSP) and/or the American standard code for information 
interchange (ASCII). According to current trends, wireless communication will 
employ higher and higher data density, and the present invention will probably be 

20 important for implementing trellis-based soft bit equalizers in any wireless 

transmission environment that is significantly dispersive, for example in a high 
density and high delay system like 16-64 QAM (quadrature amplitude modulation). 

Brief Description of the Drawings 
25 Figure 1 is a flow chart illustrating basic steps of the present method. 

Figure 2 shows the structure and interactive cooperation within the present 
invention, in the context of a wireless communication receiver. 

Figure 3 illustrates an embodiment of the present invention employing DFE 
and either RSSE or Max-Log-MAP. 
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Figure 4 depicts an aspect of an embodiment in which soft bits are computed 
symbol-by-symbol using with zero delay. 

Figure 5 illustrates an embodiment of the present invention employing zero 
delay and either RSSE or Max-Log-MAP. 
5 Figure 6 shows an embodiment of the present invention employing DFE and 

DFPSE. 

Figure 7 shows a trellis for a two-state equalizer. 
Figure 8 shows a trellis for a four-state equalizer. 

10 Best Mode for Carrying Out the Invention 

General features of a best embodiment of the present method are illustrated 
by the flow chart of Figure 1 . The method involves computing 100 a first set of soft 
bits for bits transmitted in a received signal, using a reduced-state trellis with 
forward recursion and with finite non-zero delay. Alternatively, the computation 

15 100 may advantageously utilize a Viterbi Algorithm, such as a soft output Viterbi 
algorithm (SOVA). The method also involves calculating 110 hard decisions in 
response to the received signal, and also ensuring 120 that substantially all soft bits 
are computed by employing zero-delay soft decision-making or decision-feedback 
equalization to compute a second set of soft bits. The hard decisions are used to 

20 compute the second set. 

Although Figure 1 depicts a particular sequence for the steps, different 
sequences can be used in other best mode embodiments. The sequence shown in 
Figure 1 is desirable if we want to ensure that the first set of soft bits and the second 
set of soft bits are soft bits for two different sets of bits. In other words, once the 

25 first set is computed 100, then only soft bits for the remaining bits need to be 

computed 120 for the second set. The hard decisions are calculated 110 using the 
first set of soft bits. Using the hard decisions to compute soft bits is helpful to 
reduce error propagation. 
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In some applications, it will be useful to augment the method shown in 
Figure 1, by generating further hard decisions using a traceback operation, 
checking if the further hard decisions generated using the traceback operation match 
the hard decisions already calculated in step 110 using the first set of soft bits, and 
5 replacing soft bits generated using the trellis with the further hard decisions 

generated using the traceback operation, if the further hard decisions generated from 
the traceback operation fail to match respective hard decisions already calculated in 
step 110. 

*S As shown in Figure 2, the present invention can be embodied in a wireless 

£ 10 communications receiver 295 containing a reduced-state equalizer 200. The 

t 

01 equalizer 200 is capable of providing soft bits to a decoder 290 in order to enhance 

fl reliability of channel decoding subsequent to equalization. The reduced-state 

^ equalizer includes a reduced-state trellis module 220, responsive to a received bit 

O signal on a line 260, for providing a soft bit signal on a line 280 and a hard decision 

Mb 15 signal 270 on a line 270. The equalizer also includes a gap-filler module 210, 
?2 responsive to the received bit signal and also responsive to the hard decision signal 

^ 270, for providing a second soft bit signal 292. The second soft bit signal 292 

includes soft bits that are absent from the first soft bit signal, so that the first and the 
second soft bit signals include soft bits for substantially all bits in the received bit 
20 signal 260. In other words, the gap-filler module 210 produces soft bits which will 
fill any gaps in the soft bit computation performed by the reduced-state trellis module 
220. The reduced-state trellis module 220 is configured to operate with forward 
recursion and finite non-zero delay, and the gap-filler module 210 is configured as a 
decision feedback equalizer or as a zero-delay soft decision-maker (i.e. configured 
25 for DFE or zero-delay soft decision-making). The decoder is responsive to the first 
soft bit signal 280 and the second soft bit signal 292, and is configured to provide a 
reliably decoded signal 297. Of course, inventive portions of the receiver and 
equalizer are shown in Figure 2 while other structures and signals are omitted from 
Figure 2 for the sake of clarity. 
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In this embodiment, the finite non-zero delay employed by the reduced-state 
trellis module 220 is K minus one (K-l), where K is a truncated impulse response 
length and is less than or equal to a non-truncated impulse response length L (i.e. 
K<L). For the zero-delay soft decision-making, which may be employed by the 
5 gap-filler module 210, K vanishes (i.e. K=0). Alternatively, if the gap-filler 
module 210 is configured for DFE, then the second set soft bits can be obtained 
with delay K minus two (K-2) if the trellis module 220 is configured for max-log- 
MAP. But, if the trellis module 220 is of an RSSE type then the gap-filler module 
210 can have arbitrary delay. Only forward recursion is used in the best 

10 embodiment of the present invention, and thus forward recursion is sufficient 

without the typical backward recursion, although backward recursion is possible in 
other embodiments of the present invention. 

Figure 3 is a flow chart illustrating a particular embodiment of the present 
invention, employing a DFE step 310 in order to fill gaps, and an RSSE/Max-Log- 

15 MAP step 320 to perform a reduced-state trellis calculation. Whereas Figure 1 
showed a non-zero delay computation followed by a DFE computation, Figure 3 
illustrates the DFE computation 310 occurring before the RSSE/Max-Log-MAP step 
320 (which is a non-zero delay reduced state trellis calculation). Thus, in the best 
mode embodiment shown in Figure 3, a soft bit computed in the DFE step 310 for a 

20 certain bit is replaced with a soft bit computed for that bit in the RSSE/Max-Log- 
MAP step 320 if the soft bit for the bit is computed in the RSSE/Max-Log-MAP 
step 320; otherwise the soft bit computed for the bit in the DFE step 310 will not be 
replaced and will effectively fill a gap in the results of the RSSE/Max-Log-MAP 
step 320. Notice that Figure 3 shows a feedback step 360, according to which hard 

25 symbol decisions are fed back from the RSSE/Max-Log-MAP step 320 to the DFE 
step 310. The flow chart of Figure 3 also shows an MSSE-DFE prefiltering step 
300 which precedes the other steps just discussed (MMSE stands for minimum 
mean-squared error), and a channel decoding step 340 which follows the other steps 
just discussed. Between the decoding step 340 and those other steps discussed, it is 
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useful to include a soft bit output step 330 in order to perform the previously 
described replacement of soft bits calculated in the DFE step 310 soft bits calculated 
in the RSSE/Max-Log-MAP step 320 (a soft bit output module could be included 
upstream from the decoder or within the decoder). Regardless of when, or in what 
5 order, the non-zero delay reduced state trellis calculation is performed, it is useful 
in a best mode embodiment to perform this calculation for as many bits as possible 
while maintaining acceptable implementation complexity. 

In Figure 3, regarding the feedback step 360 in which hard symbol decisions 
are fed back from the RSSE/Max-Log-MAP step 320 to the DFE step 310, the hard 
10 decisions may be computed using the max-log maximum a posteriori technique. 
Subsequently, the decision feedback equalization step 310 will be advantageously 
performed according to a rule described mathematically as follows: 

0DFE( a k)*l r k "iflkK + Z b k-j h j) I (!) 

7=1 

15 in which beta (P) represents a log symbol probability function, ak represents a 

symbol of a modulation alphabet, rk is a received signal, bk are the hard decisions, 
and hk is channel impulse response length. A decision produced by the DFE step 
310 can be used to limit the number of states involved in the RSSE/Max-Log-MAP 
step 320, but that is not necessary in a best mode embodiment. 

20 Figure 4 depicts an aspect of the invention in which soft bits are computed 

symbol-by-symbol using zero delay. First a received signal is subjected to an 
MMSE-DFE prefilter step 400, then is processed in a RSSE symbol-by-symbol 
zero-delay soft decision step 410, and finally is decoded in a channel decoder step 
420. 

25 The RSSE Symbol-by-symbol zero-delay soft decision step 410 

advantageously employs a trellis calculation, and implemented with a maximum 
complexity that is twice as complex as implementation of decision-feedback 
equalization would be. This RSSE Symbol-by-symbol zero-delay soft decision step 
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410 will preferably be performed according to a rule described mathematically as 
follows: 

P today = a k ) ~ min (5,) iM(S t ) + HS, , S M )} (2) 

5 in which beta represents a zero-delay log symbol probability function, Sk denotes a 
previous symbol decision, Sk+i denotes a current symbol decision, ak represents a 
symbol of a modulation alphabet, min is a minimizing function, mu (jli) and lambda 
(k) represent two paths connecting two states. 

Figure 5 illustrates an embodiment of the present invention employing zero 

10 delay and either RSSE or Max-Log-MAP. This Figure 5 is similar to the 

embodiment shown in Figure 3, the primary difference being that the DFE step 310 
of Figure 3 is replaced by the symbol-by-symbol zero-delay soft decision step 410 
of Figure 4. The flow chart of Figure 5 begins with a prefiltering step 500, 
followed by a symbol-by-symbol zero-delay soft decision step 510 and an 

15 RSSE/Max-Log-MAP soft decision step 520 (steps 510 and 520 can occur in either 
order), and then a soft bit output step 530 followed by a decoding step 540. 

When the non-zero delay trellis computation involves max-log-MAP, as may 
occur in step 320 of Figure 3 or step 520 of Figure 5, further efficiency is obtained 
if first and second subsets of signals are established by Euclidean partitioning so as 

20 to reduce trellis size to two states corresponding to the first and second subsets 

respectively. Then the more likely of the two states is determined, and Euclidean 
distances are computed for all paths corresponding to the more likely state; only the 
paths corresponding to the more likely state would then be used to perform trellis 
computation of soft bits in step 510 of Figure 5. A decision produced by the 

25 symbol-by-symbol zero-delay soft decision step 510 can be used to limit the number 
of states involved in the RSSE/Max-Log-MAP step 520, and that is necessary in a 
best mode embodiment of the present invention. 

Figure 6 shows another best mode embodiment of the present invention, 
employing DFE and decision feedback partial sequence equalization (DFPSE). 
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Figure 6 is, again, similar to the embodiment shown in Figure 3, the primary 
difference being that the RSSE/Max-Log-MAP step 320 of Figure 3 is replaced by a 
DFPSE step 620 which (again) involves a non-zero delay reduced state trellis 
calculation. The flow chart of Figure 6 begins with a prefiltering step 600, followed 
by a DFE step 610 and a DFPSE step 620. Steps 610 and 620 can occur in either 
order, and in either case a feedback step 660 feeds hard symbol decisions from 
DFPSE to DFE. Decisions produced by DFE can be used to reduce the number of 
states in the DFPSE trellis. The final steps in this embodiment are soft bit output 
630 followed by decoding 640. 

It should be emphasized that in all of the best mode embodiments shown in 
the figures, a non-zero delay reduced state trellis step computes soft bits and hard 
symbol decisions, either before or after a DFE/zero-delay step also computes soft 
bits. Furthermore, this DFE/zero-delay step always utilizes the hard symbol 
decisions to compute soft bits. The present invention thus ensures that, in one way 
or another, soft bits are efficiently calculated for substantially all bits transmitted in 
a received signal. 

Those skilled in the art will understand that each of the best mode 
embodiments described herein can be implemented in a variety of ways, by a variety 
of different combinations of hardware and software, and will further understand that 
the structures, steps, and interactions described herein are conceptual elements that 
likewise can be implemented by various combinations of hardware and software and 
integrated with a variety of other structures, steps, and interactions. 

Generally speaking, the present invention includes novel methods of soft bit 
computation for any reduced state equalizer. Although perhaps somewhat 
repetitive, it may be useful to now discuss further some general characteristics and 
examples of the invention in the context of EDGE at 8PSK. We shall consider soft 
bit generation algorithms for a reduced state set partitioning Max-Log-MAP 
equalizer using 8PSK as an example, although the same ideas are also applicable to 
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RSSE or DFPSE equalization. As an example we will use a 2-state Max-Log-MAP 
equalizer for EDGE 8-PSK in which J ={2,1,1,1,1}. 

Let us consider 2— state, 2-way set partitioning Max-Log-MAP soft bit 
detection. This example will have three phases, but first some preliminaries. 

5 Divide the 8PSK signal constellation {0, 1 . .7} into two subsets A and B where 
A = {0,2,4,6}, and B = {1,3,5,7}. The subsets are chosen such that the minimum 
Euclidean distance between any points in the subset is maximized, as discussed in 
the article by Eyuboglu and Qureshi. This type of partitioning is called Euclidean 
partitioning. The subset trellis for a 2-tap equalizer (K=2) is shown in Figure 7. A 

10 normal Viterbi trellis contains 8 states corresponding to 8 signal points. In the 

present case, the trellis size is reduced to two states corresponding to two sub sets A 
and B. Note from Figure 7 that there are parallel transitions merging at each node. 
Since the Euclidean distance between the points in a given subset is maximized, 
early merging paths can be reliably distinguished. A minimum phase condition is 

15 required when making such early decisions. Now we would like to incorporate 

forward Max-Log-MAP detection through this reduced state trellis. The prefiltered- 
received signal sampled at symbol rate can be written as: 

L-l 

r{k) = a 0 h 0 + ^ a k-j h j + n ^ ^ 
7=1 

Where ak is the 8PSK modulation alphabet, hk is the channel impulse response of 
20 length L, and n(k) is the thermal noise. To reduce complexity, the first K taps are 
used in the main part of the trellis, and decision feedback is used for the remaining 
taps. Then the number of states in the trellis is given by 2 K ~ l . In Figure 7, the 
number of states is restricted to 2. Referring to Figure 7, a symbol ak arrives at 
t=kT, and sr denotes the state at t=kT. The state labeling is given by Sk = ak-i and 
25 Sk contains the previous symbol decision. Furthermore, Sk+i = ak and Sk+i denotes 
the current symbol decision. 

The first phase in this 2-state, 2-way set partitioning Max-Log-MAP soft bit 
detection example begins by adding the path metric X{s k 9 s k+ i) to the old path 

13 
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metric ju(s k ) for all the paths merging at a given state Sk+i, and choosing the 
minimum distance winner path. Then store a k for this winner path and update the 
path history. This is denoted by the following equation: 

5 ) * min (*j {pin ) + A(sk > s k+\ )} ( 4 ) 

where Sk denotes all the states corresponding to previous symbol fik-i. In Figure 7, 
there are 8 branches leaving each node s k = a k _ x , and 8 branches merging at each 
node = a k . The state dependent path history contains all previous L-K 
10 decisions on a k . The path metric for each branch connecting (s k ,s k +i) is defined 
as 

2 



H s k> s k+l) = 



L 

r k ~( a k h 0 +a k-\ h \ + Yj®k-j h j) 
j=2 



2a 2 



(5) 



In equation (5), a k is obtained from the previous state s k , and a k is the path 
history stored in s k . 

15 The second phase in this example is max-log-MAP symbol detection with 1- 

symbol delay. In Figure 7, decisions on symbol a k are made with 1 -symbol delay 
at t=(k+ 1) T meaning at state . According to forward MAP criterion 
(forward MAP is used in this example but the invention is also applicable when 
backward recursion is included), it is known in the art that the state probabilities are 

20 given by: 

^ + i) = I ex P[-W^ + i) + % + i^it + 2)}] (6) 

S k+2 

in which s k+2 denotes all the paths leaving state s k +i . In the example of Figure 7, 

there are eight branches leaving each state. The summation in equation (6) is 

performed for all 8 branches leaving s k+x . To make a decision on symbol a k we 

25 need to decide whether a h belongs to subset A or subset B. First compute, 
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P(s k+l =A)= £expHMjjt + i = A) + X(s k+ i = A,s k+2 )}] (7) 

S k+2 

P(sk+1 =B)= £exp[-{M**+l =B) + A{s k+l =B,s k+2 )}] (8) 

The summation is performed for 5 t+2 = A and = B ♦ The hard decision rule to 
detect symbol a k is given by this rule: if P(s k+ i =A)> P(s k+ \ = B) , then a k 
belongs to state A. This means that the current symbol stored in state A will be the 
detected symbol. In practice it is easier to deal with log symbol probabilities for 
complexity reduction purposes. Define beta as follows: 

= A ) = - Wtofc+i = W ■ < 9 > 
The presence of exponentials in equation (6) is computationally expensive, and we 
resort to the Max-Log approximation for numerical simplification. The negative 
logarithm of a sum of exponentials can be approximated as: 

~ ln(X e~ Sj ) = min (^l > S 2 - S j ) < 10 > 

J 

Then equation (9) can be simplified as: 

=^)~ min (^j{M^+i = A) + X(s k + X =A,s k+2 )} (11) 

The third phase in this example is soft bit generation. In order to take Ml 
advantage of soft bit decoding gain, we need to convert the detected symbol bit 
probabilities to bit probabilities. Let us assume that state s k +i takes two 
possibilities A=0 and B = 1. Symbols 0 and 1 are mapped to the bits (111) and (Oil) 
respectively. Then the bit probabilities can be written as: 

PZ> 0 (0)~0 and p* 0 (i) = P(0) + P(l)«l.O and Pb 1 (0)^0 (12) 

and 

PZ ?1 (l) = P(0) + P(l)«l.O and P5 2 (0)-P(l) and Pb 2 (l) = P(Q) (13) 
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where pb t (0) denotes the probability of the i'th bit position being zero. It is clear 
that we can generate appropriate bit probabilities for the most significant bit (MSB), 
and the remaining two bit positions can have hard decisions only. Since it is 
computationally efficient to deal with log bit probabilities, we use the following 
5 definition for bit log-likelihood-ratios (LLR): 

q t =-hx(Pb f (l)/F6 *(<>)) (14) 

Then the LLR for each bit position can be written as: 

10 qo « ±oo and q x » ±oo and q 2 * (3(1)- /?(0) (15) 

In short, 2-state, 2-way set partitioning Max-Log-MAP soft bit detection can 
be described in the following phases. First, compute Euclidean distances for all the 
16-paths originating from nodes A and B. Second, for the MSB, search for a 

15 minimum distance among all 16 paths that have a zero in the MSB position, and 
repeat this process for bit 1 in the MSB position (soft information for the MSB is 
defined as the difference between these two minimum distances). Third, repeat the 
second phase for all other bit positions. 

For computational efficiency, we generate soft bit information directly from 

20 the Euclidean distances. We avoid the intermediate step of symbol probability 

generation. In this particular example, we can generate soft bit information for the 
MSB only; the other two bits remain in an initialization state causing soft decision 
failure. The present invention conveniently includes several solutions solutions that 
will remedy this 2-way set partitioning Max-Log-MAP soft bit detection failure 

25 problem. Further simplification can be obtained by calculating one Euclidean 

distance for all the parallel transitions between two nodes: first make a hard decision 
using a slicing operation, and then compute one Euclidean distance for the part 
corresponding to the hard decision. This reduces implementation complexity in both 
the Max-Log-MAP and the Viterbi part of the algorithm. 
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A first "soft decision feedback" solution to this 2-way set partitioning max- 
log-MAP soft bit detection failure problem can be described as follows. Generate 
hard decisions bk from RSSE/RS-Max-Log-MAP. Then run DFE with RSSE/RS- 
Max-Log-MAP hard decisions in the feedback part. For bits that remain in an 
5 initialization state, generate soft bit information for those remaining bits. For the 
MSB in the given example, use Max-Log-MAP. This first solution can best be 
understood by again referring to Figure 7. At state use RSSE/Max-Log-MAP 

to create soft bit information on symbol for possible bits, and to create soft 

information for the remaining bits run DFE on the received signal r k with Max-log- 

10 MAP hard decisions bk as feedback. This leads us back to equation (1) above, and 
this "soft decision feedback solution" generally corresponds to the flow chart in 
Figure 3. 

A second solution to this 2-way set partitioning max-log-MAP soft bit 
detection failure problem may be called the "symbol~by-symbol soft bit" solution, 
15 and it generally corresponds to the flow chart in Figure 4. In this solution, symbol- 
by-symbol soft bit information is generated in the trellis without any delay. 
Referring again to Figure 7, search for soft bit information for all the 16 paths 
arriving at s k+ i . Use the rule given by equation (2) above. In equation (2), to 

detect symbol = 0 , minimization is performed for the sixteen paths originating 

20 from . The complexity of this method is exactly twice that of DFE. 

A third solution to this 2-way set partitioning max-log-MAP soft bit 
detection failure problem may be called the "improved soft bit" solution, and it 
generally corresponds to the flow chart in Figure 5. In this solution, we utilize the 
improvements from both the first "soft decision feedback" solution, and from the 
25 second "symbol-by-symbol soft bit" solution. Begin by generating possible soft bit 
information using Max-Log-MAP. Then use the second "symbol-by-symbol soft 
bit" solution to generate soft information for the remaining bits. Referring again to 
Figure 7, at state store the Euclidean distance for all the 16 paths. During the 
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next symbol interval (i.e. at state s^+i), generate soft bit information for the MSB. 

Then generate soft information for the remaining bits by searching through the 16 
paths stored in the previous symbol interval. To further reduce the complexity of 
this "improved soft bit" solution, Max-Log-MAP detects the symbol a^_\ at state 
5 Sfc . Then, store the 8-paths originating from = ctk-\ on *y> instead of all 16- 
paths. We only need to search through these 8-paths for computing the soft bit 
information. Instead of using max-log-MAP, one can use an RSSE/MLSE soft 
output Viterbi algorithm (SOVA) to compute soft bit information. However, if soft 
Cf bit computation is not possible for a particular bit, then we can replace it using the 

10 "soft decision feedback solution" or using zero-delay. 
m A fourth solution to this 2-way set partitioning max-log-MAP soft bit 

^ detection failure problem may be called the "max-log-MAP with L symbol delay" 

solution. So far, we have implemented max-log-MAP with K symbol delay (i.e. 
□ one symbol delay for the 2-state case shown in Figure 7). We can get some extra 

2 15 gain by delaying max-log-MAP decisions by an amount greater than the K symbol 
S delay. 

.was A fifth solution to this 2-way set partitioning max-log-MAP soft bit detection 

failure problem may be called the "solutions one to four plus traceback" solution. It 
is generally believed that the hard decisions given by the traceback operation contain 
20 fewer errors than solutions 1-4. Hence, we have proposed, in the present invention, 
to generate hard decisions from traceback, and check if these decisions match the 
hard decisions from solutions 1-4. If not, replace the soft bit information from 
solutions 1-4 with the traceback hard decisions. 

Up until now, the example we have been considering has involved a 2-state 
25 Max-Log-MAP equalizer for EDGE 8-PSK in which J = {2, 1 , 1 , 1 , 1 } , and we have 
been considering 2-way set partitioning Max-Log-MAP soft bit detection. But 
suppose we have a 4-state Max-Log-MAP equalizer, and let us consider 2-way set 
partitioning and soft bit detection by analogy with the two-state example. Instead of 
Figure 7, we must now refer to the 3-tap, 4-state trellis diagram of Figure 8. In 

18 
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Figure 8, each state s k can have four possibilities AA, AB, BA, BB where A takes 
one of the four symbols from the subset {0,2,4,6}, and B takes values from the 
subset {1,3,5,7}. The state labeling is given by: 

Sk =(a*-2.«Jfc-l) 311(1 s k+l =(a k _x,a k ) (16) 

There are two phases in this 4-state, 2-way set partitioning Max-Log-MAP 
soft bit detection example. The first step utilizes an equation which is a slight 
revision of equation (4): 

fj(s k+ i) « mm is > k) {ju(s' k ) + A(s' k ,s k+ i)} (17) 



Equation (17) denotes that for all the paths merging at a given state s k+l , add the 
path metric A(s' k ,s k+1 ) to the old path metric ju(s' k ) , and choose the minimum 
15 distance winner path. We introduce the notation s' k to denote all the paths 

originating from nodes s' k and merging at node s k+ i . The path metric is defined 
by an equation which is a slight revision of equation (5): 



r k -(a k h 0 + Y< a k-j h j + lL a k-j h j) 
7=1 J=K 



2a 2 



(18) 



where a k is obtained from the previous state s' k , and a k is the path history stored 
20 in 4 . From Figure 8, all the paths merging at s k+l = AA can have transitions 
originating from s' k = AA and s' k = BA only. In this case equation (17) can be 
written as: 

M{Sk+l — AA) « min (j;) {fi(s' k ) + X{s' k , AA)} (19) 
where s' k takes two possibilities (A A and BA). After determining the winner path, 
25 update the state-dependent path history vector a k . 

19 
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The second phase of this 4-state, 2-way set partitioning Max-Log-MAP soft 
bit detection example requires that the state probabilities are given by: 

s ' 

5 according to forward MAP criteria. For K= 3, as is evident from Figure 8, 
equation (20) can be written as: 

P(s k+2 ) = ^exp[-{M(s k+2 ) + Hs k+2 ,s'k+3)}] ( 21 > 

In equation (21), s' k+3 denotes all the paths leaving nodes s k+2 a°d terminating at 
10 4+3 . For example, if s k+2 = AA , then s' M takes two possibilities: AA and BA. 
There are a total of eight branches leaving s k+2 = AA and terminating at s' k+3 = AA 
or s' k+3 = BA . Accordingly: 

P(s k+2 =AA)= 5>xp[-{M^+2 = AA) + X(s k+2 = AA,s' k+3 )}] ( 22 > 



5 k+3 



P(s k+2 =BA)= X ex PHM^ + 2 = BA) + X{s k + 2 =544+3))] (23) 
15 Then we find that: 

Similarly: 

P(s k+2 =B) = P(s k+2 = AB) + P(s k+2 = 55) (25) 

20 The final decision on symbol a k would be one of the two states A or B, whichever 
is more probable. Since we are interested in the soft bit information, we need to 
compute bit level soft information. This is done exactly same way as described 
above in the two-state example. Moreover, equations (21) - (24) can be generalized 
to any set partitioning type, and to an arbitrary number of taps, in a straightforward 

25 way. 
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Although this invention has been shown and described with respect to best 
mode embodiments and specific examples thereof, it should be understood by those 
skilled in the art that the foregoing and numerous other changes, omissions and 
additions in form and detail may be made therein without departing from the spirit 
and scope of the present invention. 
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